Papers with policy gradient method
Policy Gradient as a Proxy for Dynamic Oracles in Constituency Parsing (P18-2)
Copied to clipboard
| Challenge: | Existing models for constituency parsing are transition based, decomposing production of each parse tree into a sequence of action decisions. |
| Approach: | They propose to use a policy gradient method to reduce exposure bias by allowing exploration during training. |
| Outcome: | The proposed method outperforms static oracle likelihood training on four constituency parsers in three languages on four parser-agnostic languages. |
Adversarial Grammatical Error Correction (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Experimental results show that adversarial-GEC can achieve competitive GEC quality compared to NMT-based baselines. |
| Approach: | They propose an adversarial approach to Grammatical Error Correction using a transformer-based model and a sentence-pair classification model. |
| Outcome: | The proposed approach achieves competitive GEC quality compared to baselines. |
Tutoring Helps Students Learn Better: Improving Knowledge Distillation for BERT with Tutor Network (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge distillation approaches for language models have overlooked the difficulty of training examples. |
| Approach: | They propose a framework that controls difficulty of training examples during pre-training by a tutor network. |
| Outcome: | The proposed framework outperforms state-of-the-art KD methods with student models on the GLUE benchmark. |